Measuring Variations of Mimicry by Means of Prosodic Cues in Task-Based Scenarios and Conversational Speech

Here, we address the measurement of mimicry, that is when speakers’ speech variations look like parallel patterns. As a definition of mimicry, we often read in the literature description such as mimicry is “The situation where the observed behaviours of two inter-actants although dissimilar at the start of the interaction are moving towards behavioral matching”. These types of descriptions imply that mimicry is a linear phenomenon and that speakers tend to imitate over time. However, it can be assumed, especially when studying spontaneous speech, that there are rather phases of mimicry and non-mimicry and that mimicry should be rather investigated as a dynamic phenomenon. As mimicry has been considered so far as a linear phenomenon, metrics developed to measure this phenomenon failed to capture its temporal dynamics. In this work then, we propose to measure mimicry on the whole interaction but also at certain points of the conversation

Providing Objective Metrics of Team Communication Skills via Interpersonal coordination Mechanisms

Being able to communicate efficiently has been acknowledged as a vital skill in many different domains. In particular, team communication skills are of key importance in the operation of complex machinery such as aircrafts, maritime vessels and such other, highly-specialized, civilian or military vehicles, as well as the performance of complex tasks in the medical domain. In this paper, we propose to use prosodic accommodation and turn- taking organisation to provide objective metrics of communica- tion skills. To do this, human-factors evaluations, via a coordi- nation Demand Analysis (CDA), were used in conjunction with a dynamic model of prosodic accommodation and turn-taking organisation. Using conversational speech from airline pilots involved in a collaborative task (decision-making exercise), our study reveals that interpersonal coordination mechanisms are indicative of human evaluation of pilots’ communication skills. We discuss our results in terms of relevance for training simu- lation for personnel in safety or mission critical environment

Investigating Automatic Measurements of Prosodic Accommodation and Its Dynamics in Social Interaction

Spoken dialogue systems are increasingly being used to facilitate and enhance human communication. While these interactive systems can process the linguistic aspects of human communication, they are not yet capable of processing the complex dynamics involved in social interaction, such as the adaptation on the part of interlocutors. Providing interactive systems with the capacity to process and exhibit this accommodation could however improve their efficiency and make machines more socially-competent interactants. At present, no automatic system is available to process prosodic accommodation, nor do any clear measures exist that quantify its dynamic manifestation. While it can be observed to be a monotonically manifest property, it is our hypotheses that it evolves dynamically with functional social aspects. In this paper, we propose an automatic system for its measurement and the capture of its dynamic manifestation. We investigate the evolution of prosodic accommodation in 41 Japanese dyadic telephone conversations and discuss its manifestation in relation to its functions in social interaction. Overall, our study shows that prosodic accommodation changes dynamically over the course of a conversation and across conversations, and that these dynamics inform about the naturalness of the conversation flow, the speakers’ degree of involvement and their affinity in the conversation

Using Hotspots as a Novel Method for Accessing Key Events in a Large Multi-Modal Corpus

n 2009 we created the D64 corpus, a multi-modal corpus which consists of roughly eight hours of natural, non-directed spontaneous interaction in an informal setting. Five participants feature in the recordings and their conversations were captured by microphones (room, body mounted and head mounted), video cameras and a motion capture system. The large amount of video, audio and motion capture material made it necessary to structure and make available the corpus in such a way that it is easy to browse and query for various types of data that we term primary, secondary and tertiary. While users are able to make simple and highly structured searches, we discuss the use of conversational hotspots as a method of searching the data for key events in the corpus; thus enabling a user to obtain a broad overview of the data. In this paper we present an approach to structuring and presenting a multi-modal corpus based on our experience with the D64 corpus that is accessible over the web, incorporates an interactive front-end and is open to all interested researchers and student

Speech & Multimodal Resources: the Herme Database of Spontaneous Multimodal Human-Robot Dialogues

This paper presents methodologies and tools for language resource (LR) construction. It describes a database of interactive speech collected over a three-month period at the Science Gallery in Dublin, where visitors could take part in a conversation with a robot. The system collected samples of informal, chatty dialogue – normally difficult to capture under laboratory conditions for human-human dialogue, and particularly so for human-machine interaction. The conversations were based on a script followed by the robot consisting largely of social chat with some task-based elements. The interactions were audio-visually recorded using several cameras together with microphones. As part of the conversation the participants were asked to sign a consent form giving permission to use their data for human-machine interaction research. The multimodal corpus will be made available to interested researchers and the technology developed during the three-month exhibition is being extended for use in education and assisted-living applications

Investigating Fine Temporal Dynamics of Prosodic and Lexical Accommodation

Conversational interaction is a dynamic activity in which participants engage in the construction of meaning and in establishing and maintaining social relationships. Lexical and prosodic accommodation have been observed in many studies as contributing importantly to these dimensions of social interaction. However, while previous works have considered accommodation mechanisms at global levels (for whole conversations, halves and thirds of conversations), this work investigates their evolution through repeated analysis at time intervals of increasing granularity to analyze the dynamics of alignment in a spoken language corpus. Results show that the levels of both prosodic and lexical accommodation fluctuate several times over the course of a conversation

StrokeCog Markov Model Projected Prevalent and Incident Cases of Stroke and Poststroke Cognitive Impairment to 2035 in Ireland

Background and Purpose: Cognitive impairment no dementia (CIND) and dementia are common stroke outcomes, with significant health and societal implications for aging populations. These outcomes are not included in current epidemiological models. We aimed to develop an epidemiological model to project incidence and prevalence of stroke, poststroke CIND and dementia, and life expectancy, in Ireland to 2035, informing policy and service planning. Methods: We developed a probabilistic Markov model (the StrokeCog model) applied to the Irish population aged 40 to 89 years to 2035. Data sources included official population and hospital-episode statistics, longitudinal cohort studies, and published estimates. Key assumptions were varied in sensitivity analysis. Results were externally validated against independent sources. The model tracks poststroke progression into health states characterized by no cognitive impairment, CIND, dementia, disability, stroke recurrence, and death. Results: We projected 69 051 people with prevalent stroke in Ireland in 2035 (22.0 per 1000 population [95% CI, 20.8-23.1]), with 25 274 (8.0 per 1000 population [95% CI, 7.1-9.0]) of those projected to have poststroke CIND, and 12 442 having poststroke dementia (4.0 per 1000 population [95% CI, 3.2-4.8]). We projected 8725 annual incident strokes in 2035 (2.8 per 1000 population [95% CI, 2.7-2.9]), with 3832 of these having CIND (1.2 per 1000 population [95% CI, 1.1-1.3]), and 1715 with dementia (0.5 per 1000 population [95% CI, 0.5-0.6]). Life expectancy for stroke survivors at age 50 was 23.4 years (95% CI, 22.3-24.5) for women and 20.7 (95% CI, 19.5-21.9) for men. Conclusions: This novel epidemiological model of stroke, poststroke CIND, and dementia draws on the best available evidence. Sensitivity analysis indicated that findings were robust to assumptions, and where there was uncertainty a conservative approach was taken. The StrokeCog model is a useful tool for service planning and cost-effectiveness analysis and is available for adaptation to other national contexts.Peer reviewe

Providing Objective Metrics of Team Communication Skills via Interpersonal Coordination Mechanisms

Being able to communicate efficiently has been acknowledged as a vital skill in many different domains. In particular, team communication skills are of key importance in the operation of complex machinery such as aircrafts, maritime vessels and such other, highly-specialized, civilian or military vehicles, as well as the performance of complex tasks in the medical domain. In this paper, we propose to use prosodic accommodation and turn- taking organisation to provide objective metrics of communica- tion skills. To do this, human-factors evaluations, via a coordi- nation Demand Analysis (CDA), were used in conjunction with a dynamic model of prosodic accommodation and turn-taking organisation. Using conversational speech from airline pilots involved in a collaborative task (decision-making exercise), our study reveals that interpersonal coordination mechanisms are indicative of human evaluation of pilots’ communication skills. We discuss our results in terms of relevance for training simu- lation for personnel in safety or mission critical environment

Les anomalies de la fréquence fondamentale chez le locuteur Parkinsonien : contraste entre les effets respectifs de l'hypodopaminergie dûe à la maladie de Parkinson et de l'apport thérapeutique par L-Dopa

International audienceVoice and speech impairments are frequent in Parkinson's disease, particularly when the disease is at an advanced stage. These impairments affect spoken communication and may become a serious disability for someone with Parkinson's disease. Many studies based on auditory-perceptual or acoustic methods have been carried out to characterize dysarthria. The heterogeneity of evaluation methods and experimental bias however make results difficult to understand. For instance, in terms of phonatory impairments and with regard to F0, results are contradictory: PD speech may be characterized by either higher F0 or lower F0 compared to control subjects, or there may be no difference at all between the two population. In this study, we aim to provide a conceptual and methodological framework which allows for interpreting the results obtained from 44 speakers (29 PD and 15 control subjects) in relation to physiological (gender, age, PD subjects' pharmacologic state) and linguistic (speech production tasks) constraints. For the present corpus, we did not observe any F0 mean difference between the two groups. Our results however reveal a signifi - cant increase in F0 mean in PD subjects under L-dopa. We assume a double and opposite effect on F0 mean during drug withdrawal: low sub-glottal pressure, due to PD, results in a decrease in F0, while laryngeal rigidity leads to an increase in F0. These two effects thus mutually annihilate. Under L-Dopa, however, the drug effect increases sub-glottal pressure, which combined with an increase in F0 due to rigidity, leads to a global increase in F0.Objectif : les troubles de la voix et de la parole sont fréquents dans la maladie de Parkinson (MDP), en particulier lorsque la maladie est installée depuis plusieurs années. Ces troubles ont un impact sur la communication parlée, ce qui peut entrainer un handicap important pour les personnes qui en souffrent. De nombreuses études s'appuyant sur des méthodes perceptives ou acoustiques ont été menées pour tenter de caractériser cette dysarthrie parkinsonienne. Or, l'hétérogénéité des méthodes d'évaluation et les biais expérimentaux aboutissent à des résultats difficiles à interpréter. Dans le cadre de notre étude, nous nous intéressons spécifiquement aux troubles phonatoires et particulièrement à la fréquence fondamentale (F0). Or, la littérature laisse apparaître des résultats contradictoires : élévation, baisse ou pas de différence de F0 dans la MDP par rapport à une population de sujets contrôle. L'objet de ce travail est donc de fournir un cadre conceptuel et méthodologique permettant d'expliquer les résultats obtenus sur la F0 moyenne de locuteurs MDP.Méthode : le corpus de l'étude est constitué de 44 locuteurs (29 patients MDP et 15 sujets témoins) en contrôlant des contraintes physiologiques (sexe, âge, état pharmacologique des patients) et linguistiques (tâches de production de parole).Résultats : sur ce corpus, nous n'avons pas observé de différence de F0 moyenne entre les patients MDP sans médicament antiparkinsonien et la population contrôle. En revanche, une augmentation significative a été mise en évidence lors de la prise de L-dopa.Conclusion : nous émettons l'hypothèse d'un double effet opposé sur la F0 dans la situation de sevrage médicamenteux. La pression-sous glottique basse due à la MDP induit une diminution de F0 tandis que la rigidité laryngée entraîne une élévation de F0, ces deux effets s'annihilent. En revanche, en situation de prise de L-dopa, l'effet médicamenteux restituant une pression-sous glottique plus élevée entraine donc, concomitant à l'augmentation de hauteur par rigidité, une hausse de la F0

Contribution à la modélisation de la phonation parkinsonienne : contraintes physiologiques et linguistiques

International audienceAn important characteristic of speech in Parkinson's disease (PD) is a vocal disorder and/or a melodic dysfunction. The fundamental frequency (F0) thus appears as an important parameter to study in this context. However, the literature reveals conflicting results: increased, decreased or no difference of F0 in PD speakers compared to healthy speakers. These contradictions are related to a lack of model in PD dysarthria. The purpose of this work is to provide a conceptual and methodological framework to explain the results of 44 speakers (29 patients and 15 control subjects) including physiological constraints (gender, age, pharmacological status of patients) and language constraints (speaking tasks).Des caractéristiques importantes de la parole dans la maladie de Parkinson (MDP) sont un trouble phonatoire et/ou une déstructuration mélodique. La fréquence fondamentale (F0) apparait donc comme un paramètre important à étudier dans ce cadre. Or, la littérature laisse apparaître des résultats contradictoires : élévation, baisse ou pas de différence de F0 dans la MDP par rapport à une population de sujets contrôle. Ces contradictions sont liées à la non prise en compte de modèle de phonation dans la MDP. L'objet de ce travail est donc de fournir un cadre conceptuel et méthodologique permettant d'expliquer les résultats obtenus sur la F0 de 44 locuteurs (29 patients MDP et 15 sujets témoins) notamment en contrôlant des contraintes physiologiques (sexe, âge, état pharmacologique des patients) et linguistiques (tâches de production de parole)